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Abstract 

The approximation of a stationary time-series by finite order autoregressive (AR) and moving averages 
(MA) is a problem that occurs in many applications. In this paper we study asymptotic behavior of the 
spectral density of finite order approximations of wide sense stationary time series. It is shown that when 
the on the spectral density is non-vanishing in [—it, tt] and the covariance is summable, the spectral density 
of the approximating autoregressive sequence converges at the origin. Under additional mild conditions on 
the coefficients of the Wold decomposition it is also shown that the spectral densities of both moving average 
and autoregressive approximations converge in L2 as the order of approximation increases. 

Keywords: Wide sense stationary time series, autoregressive estimate, moving average estimate, spectral den- 
sity. Wold decomposition, time average variance constant 

Short-title: Convergence of spectral density 



1 Introduction 

The linear estimation of a time series from a finite number of past observations is a problem encountered in 
many fields of application. This paper deals with the spectral properties of finite order linear approximations 
for a regular zero-mean, real-valued wide sense stationary (WSS) sequence with finite second moment. From 
the Wold decomposition ( |[T0ll24l '). it follows that such a process can be expressed as an infinite weighted sum 
of unit variance, uncorrelated random variables called the innovation process. This is a moving average (MA) 
type model for the process. By a one-step application of the projection theorem, on the other hand, one obtains 
an infinite order autoregressive model of the process, wherein the process is expressed as a weighted sum of all 
its past values, plus the current value of the innovation process. 
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The problem of approximating an infinite-order process and its spectral density using a finite order model 
has had a long history of research (191) and a detailed survey of the key results in this area is available in 
|[T6l . A number of problems have been studied related to choosing the optimal order for the approximation 
for parsimonious modeling as in the work of Akaike through the well known Akaike Information Criterion 
(AIC, Q) and the Final Prediction Error criterion (HI, 121). Related issues are the estimation of the spectral 
density through autoregressive (AR) models in the work of Parzen ( ifT/lfTSlfTQl ) or that of Priestley ([6 ]) which 
compares the performance of autoregressive models to that of window based spectral estimators. Among the 
papers that address the asymptotic behaviour of finite-order estimates as the model order goes to infinity, the 
results of 151 are noteworthy. In the works of Pourahmadi ( l22l[TTll23l ). the nature and rates of convergence of 
the autoregressive coefficients have been discussed for univariate and multivariate stochastic processes. Similar 
results on the rate of convergence have been considered in lT3l and 12T1 . 

While there are a number of works concerned with the optimal model order; few results are available on 
the convergence of the spectral density of the approximating finite order autoregressive process. The main 
motivation of this paper is to study the asymptotic behavior of the spectral density of finite order approxima- 
tion models, as the order approaches infinity and to obtain conditions under which the spectral density of the 
approximation converges to the true spectral density. 

The value of the spectral density at the origin plays a special role in the invariance principles for stationary 
ergodic sequences. Let Xn = ^ Z^ILi where {Xk} is a WSS ergodic process. Let the value of the spectral 
density be finite at the origin and denote it by V^. This quantity is called the Time Average Variance Constant 
(TAVC) of the process ( l25l ) and according to the central limit theorem due to Ibragimov and Linnik ( lT4l ). 

V^(X„-^) ^iv(o,r2) 

where =^ denotes convergence in distribution. 

plays an important role in steady-state simulations, where the objective is to compute the limit 
lim„_s.oo Xn when it exists (lH). One way of estimating is by windowed estimates based on finite order 
autoregressive approximations of the observed stationary process. Instead of directly estimating the moments, 
one obtains approximations for the spectral density at the origin. 

In this paper it is shown that when the spectral density of the process is strictly non-vanishing in (— ^ , ^] and 
its covariance sequence is in ii, the spectral density of the approximating autoregressive sequence converges at 
the origin, as the order of approximation goes to infinity. It is further shown that under the additional condition 
that the coefficients of the Wold decomposition of the original process belong to ^i, spectral densities of both 
the moving average and the autoregressive estimates converge in L2. 



2 Preliminaries 

Consider a zero-mean, regular, discrete time, real-valued WSS stochastic process {Xn}nez defined on a prob- 
ability space ($7, ^, P). Let L2(P) denote the Hilbert space of random variables with finite second moment 
with the inner product ]E[-, •] defined thereon. Two zero mear0 random variables X,Y ^ ^2(1*) are said to be 
orthogonal if lE[Xy] = 0. Let Rk = ]E[X„_|_fcX„] denote the covariance sequence. 

The spectral density 5(A) is defined as the Fourier transform of the covariance sequence. 

5(A) = ^ i^fce-2"*^^ AG 

The covariance sequence can be recovered from the spectral density using the inverse Fourier Transform. 

1 

Ri^= 5(A)e2^^^^dA, G Z 

'in this paper it is always assumed that the underlying random variables are of zero mean 
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Forn G Z define the space i7„ = linear span of {X„, X„„i, X„_2, . . .}, i.e., tlie Hilbert subspace generated 
by the closure of the linear combinations of X„ and all its past values at time n. 

Let y be a random variable defined on L2(P). Define ]E[y|i?„] as the projection of Y onto the space 
with respect to the inner-product ]E[-, •]. Then ]E[y|//„] is the minimum mean squared error (MMSE) linear 
estimate of Y given Hn. 

Define {vn]nez as the innovation process associated with {Xn}nez, i-e., 

l^n = Xn — ]E[X„|i/„„i] 

with IE[m„] = and IE[i/„z/fc] = a'^6n-k^ where 6^ denotes the &onecker delta. Without loss of generality it 
is assumed that o"^ = 1. By construction, Vn is orthogonal to H^-i, for all k < n. The sequence {i'k}kez,k<n 
spans the subspace Hn and constitutes an orthogonal basis in L2{TP) for the latter. Note that ]E[X„|//„_i] 
corresponds to the MMSE linear estimate of X„ given the space Hn-i and it can therefore be written as a 
linear combination of all the past values of Xn as follows: 

oo 

E[X„|//„„i] = ''^bkXn-k (2.1) 

k=l 

where the bkS minimize the mean squared error. The corresponding mean squared error is 

oo 

n{Xn-yZ^kXn-k?] = niXn-nXn\Hn-l]f] 

= lE[ul] = 1 for all n (2.2) 



k=l 



Wi[Xn\Hn^i] is thus an infinite-order autoregressive estimate of Xn- On the other hand, being a wide sense 
stationary process, Xn may be expressed in terms of its Wold decomposition as follows ( |[T0ll24l ). 

oo 

Xn = ^ aki^n-k for all ?i E Z (2.3) 

A;=0 
oo 

with I ttfc p < oo and = 1 . 

k=0 

This gives an infinite order moving average representation of Xn- The coefficients {6^} and {ak} ai^e related to 
each other as follows. For each k, 

k 

h = '^ajbk-j (2.4) 



From (12.31) and the properties of the innovations sequence it is seen that: 

oo 

Rk = ^ CLnttn-k 
n=0 

Therefore, if the spectral density is finite at the origin then: 

oo / oo \ ^ 

^ -Rfc = ^ ^ andn-k = ( "fc 1 < 

fcez fceZn=o \fc=o / 

Let p be a positive integer, and define the space Hn = linear span of {Xn, Xn~i, Xn~2, ■ ■ ■ Xn~p+i}, i.e., 
the space of all linear combinations of the p most recent values of the sequence at time n, including Xn, and all 
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their limits when they exist. Hn is then a closed Hilbert subspace of Hn, for all p and n. Clearly, H!l^ = H^, 
by definition. 

Define X„ p as the MMSE autoregressive approximation of X„ of order p. Then X„ p is a Unear combina- 
tion of {Xn-i,Xn-2, Xn-p} and is given by 



(2.5) 



k=l 



where the coefficients 6fc p minimize the error ]E[(X„ — Y^f.^i bk^pXri-k)"^]- The coefficients 6^,- p can be ob- 
tained as solutions to the modified Yule- Walker equations ( ll20l[T5l ) given by: 



B„ = 



■p ' p 



where 



Rr. 



( Ro Ri 
Ri Ro 

\Rp-i Rp-2 
rp = [Ri R2 . 



Rp~i\ 

Rp-2 

Ro J 



Rn 



and 



Bp — [61, p ... 6p,p] 



3 Moving average approximations of regular stationary sequences 

Consider a moving average approximation of X„ of order p, constructed using the innovation sequence {un}. 
Note by assumption, Vai-[M„] = al = I for all n E N. We first state some known facts without proof. 



F 3.1. The best pth order moving average approximation of Xn is given by Xn,p = X]fc=o ^^^'^ 



n—k- 



The next result is related to the mean square convergence of the p-th order approximation that readily 
follows from the properties of the innovation process and the Wold decomposition. 

F 3.2. As p ^ 00, Xn^p converges to X„ in quadratic mean. 



3.1 Convergence of the Spectral Density in L2 

For all functions F : (-i, ^] ^ C such that jK |F(A)pdA < 00, define 



to be the L2 norm as follows: 



l|i^(A)|| 



\FiX)\'dX 



Proposition 3.1. Under the condition X^^q < 00 where {a^} correspond to the coefficients in the Wold 
decomposition, the spectral density ofXn,p converges uniformly in L2 to that of Xn as p ^ 00. 

Proof. Let X^^qI'^'^I ~ ^- follows from the properties of the innovation process, that the covariance 
sequence of is {Rk} where 

00 

Rk = ^ aiai-k 
1=0 
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The spectral density S'x(A) is given by 



2TTikX 



fcez /=o 



2nik\ 



where for M,N gN 



and 



AT 



l=M 



Similarly, the spectral density of Xn,p is given by 



Now consider 



where 



Then, 



and similarly, 



||5x(A)-%^(A)|| 

= ||^~(A)^-(-A)-^^(AK(-A)|| 
= \\{Al{\)A^_,,{-X)+A^_,,{X)A^{-X)\\ 
< \\{AliX)A^_,,{-X)\\ + \\A^_,,{X)A^{-X)\\ 



hip,X)dX 



+ 



/2(p,A)dA 



/i(p,A) = |^g(AM^+i(-A)p 
h{p,X) = \A^_,,iX)A^i-X)\' 

h{p,X) = |^g(A)|Vr+i(-A)l' 



/2(P,A)<5M ^ \ak\ 



Since the sum YlT=o \ ^k\ is non-decreasing and converges to 5 < oo, for any given e > 0, there exists a 
positive integer p such that J2T=p+i I'^kl < jg, so that 

,2 



(3.2) 
(3.3) 



Combining (Il2l) and in ([XB yields 

ll'5x(A)-5^^(A)|| < 

Thus, for any e > 0, there exists a positive integer p such that IIS'x(A) — S-^ (A)|| < e for all A. Therefore, 



2 

—dX 

1 4 

2 



+ 



2 

— dA 

1 4 

2 



5-5% (A) converges uniformly in L2 to S'x(A) as p — 00. 



□ 



4 Finite AR approximations of regular stationary sequences 

We begin by looking at the asymptotic behaviour of the autoregressive approximation given by ( 12.51 ). Define 
Un^p as the error in estimation corresponding to the AR-p approximation of {Xn}, i.e.. 



Note that for all p G N 



Xn — JE[Xn\Hl^_i] 

niXn -nXn\HP_,])\Hn-l] 
nXn\Hn^l] -nXn\K-l\ 
{Xn-nXn\H^-l]) 
{Xn — 'E[Xn\Hn~l]) 



(4.1) 



Let TE[un,p\Hn^i] 



^n,p 



Then 



i^n,p - i^n and 



n,p 







As Xn is a WSS sequence in n and Vn^p is constructed as a linear combination of Xn, Xn- 



., Xn~p whose 



coefficients do not depend on n, it follows that Un.p is also a WSS sequence in n. The variance of ;/„ p, then, 
is only a function of p. Let Var[i/„ p] = dp. By (14.11) . and the fact that z/„ is orthogonal to the subspace Hn-i 
(and hence to e„ p) we obtain: 

al = Var[i/„] + Var[e„,p] 
= 1 + Var[e,,,p] 
= 1 + E[e2j 
> 1 for all p G N 

Note that for any q, p £ N such that q > p, H^_i C H^_i- It follows then, that E[X„|if^_ J is at least as 
good a linear estimate of X„ as ]E[X„|i?^_^], in terms of mean squared error. Therefore, 

niXn - nXnlK^i]?] < mXn " 1E[X„ | //^ J )2] 
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and hence 



^2 ^ J2 



Hence, the sequence cTp is a non-increasing sequence in p, bounded below by 1 and must therefore has a limit 
as p — oo that is bounded from below by 1. It can be shown that the limit is in fact equal to 1. 

Remark 4.1. As p ^ oo, ]E[X„|ff^_^] — )• ]E[X„|ff„„i] and i'n,p in quadratic mean. 

The proof is a direct application of |[TOl Lemma 3.1(b)]. 

4.1 Convergence of the Spectral Density at the Origin 

We now study conditions under which the spectral density of the finite order AR approximation converges at the 
origin. As mentioned 5(0) is refeiTcd to as the Time Average Variance Constant (TAVC) and plays an important 
role in simulations. Throughout the following it is assumed that the spectral density ^(A) is non-vanishing in 
5] there exists e > such that 5(A) > e for all A G ^] ■ 
We start with the following lemma on the pointwise convergence of the coefficients bk^p ( lfT2l Proposition 
3.1]). 

Lemma 4.1. As p ^ 00, b^^p — )• bf^ for each /c G N. 

Proof. For each p G N and A; G {1, . . . ,p} define 6o,p = 1 and b^^p = —bk,p- For any p, Vn is given by 

p 

^n,p — ^ ^ bj^^pX^i—k 
k=0 

The above can be written in matrix form as = i?X where ly = [i^nfi • • • T^n,pY , X = [X„ • • • Xn-pY and 
i? is a lower triangular matrix whose first column is [6o,p • • • &p,p]- The matrix B is invertible with inverse A 
which satisfies X = Av. The inverse A is lower triangular with first column [ao,p • • • CLp^p] and the elements j 
satisfy for all n G N 

p 

Xn = ^ ak,p'^n-k,p-k (4.2) 
k=0 

and for p > A; 

k 

bk,p = — ^ '^j,pbk-j,p-j (4-3) 

By definition, „ and i^m,m are orthogonal to each other for n 7^ m and hence (14.21 ) provides an orthogonal 
decomposition of X„. Therefore, 

JE[Xni^n-k,p-k] = ak,pcrp_k (4.4) 

From (12.31 ). on the other hand, we have 

JE[XnUn-k] = ak (4.5) 
However, by remark l4~n f„ p — )• f„ in quadratic mean. It then follows from (14.41) and (14.51) that for all fc G N 

lim |afc,pC7p_fc - Ofcl 
p— ^00 ^ 

^Jim \'^[Xnl^n-k,p-k - Xnl^n-k]\ (^4 5) 



p— >oo 



< lim ]EK]]E[(i/„„fc,p_fe - u^^kV] 

p— ^oo ' 
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and hence, 



lim ak,p = ttk (4.7) 



Finally, to show the pointwise convergence of bk,p (and therefore that of bk,p) first observe that 6o,p = bo = 1 
holds for all p. Let 



lim bj „ = —bj 

p— >-oo 



for all j < k. Then, using the recursive relation given by (14.31 ) and comparing with (12.4b . one obtains 

lim fefc+i,p = -h+i 

p— >-co 

Therefore, by the principle of mathematical induction, as p — oo, bf^ p — )• —bk', i.e., bk^p — )■ bk for each 

ken. ' ' □ 

Next, we present a key result on the summability of the autoregressive coefficients known as Baxter's 
inequality 15] Theorem 2.2]). 

Let {Xn} be a WSS process with spectral density function 5'x(A) > and let Xn,p be the p-th order 
MMSE linear predictor of Xn, defined by 12.51 and let dp be the corresponding mean squared error. Let {fejt} be 
the limits of the coefficients {&fc,p} and let cr^ > be the limit of dp as p — )• oo (in our case fi^ = 1 by remai^k 
14.1b . Define the sequence {ui^ p} as n^ ^ = — and let {Uk} be the limit of {n^ p}. Then, the theorem is 
stated as follows. 

Theorem 4.1. Baxter's Inequality: If Sx{^) is a positive continuous function whose Fourier coefficients have 
7 moments, then there exists an integer N > and a constant c > 0, both depending only on S'x(A) such that 
for all p>N, 

P CO 

j^(2^ + F)|nfc,p-C/fc| <c {2^ + k^)\Uk\ 

k=l k=p+l 

Note that the Fourier coefficients of the spectral density are the elements of the covariance sequence {Rk}- 
The above theorem can be used to establish the following lemma on the convergence of the coefficients 

{bk,p} as p oo. 

Lemma 4.2. When the spectral density of {Xn} is strictly positive in X G (~^) i^^^ covariance se- 
quence is in ii, i.e., 

Y\Rk\ < oo 

then 

p 

lim \bk,p -bk\=0 
k=l 

Proof. The proof is a simple application of Baxter's inequality. Note that when the covariance sequence is in 
£i, the spectral density is continuous. Pointwise convergence of the 6fc_p to bk for each k follows from lemma 
14.11 Moreover, summability of the covariance sequence also imply that the sequence has a finite 0-th moment 
(7 = in Theorem 14. lb . It then follows from Theorem 14. II that under the assumption that the spectral density 
of {Xn} is strictly positive in A G (— |) ^] > there exists a positive integer N and a constant c > such that 



p 

E 

k=l 



Ok,p 



<c J2 (4.8) 

k=p+l 
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for all p > N. Since the covariance sequence has been assumed to be in £i, the sequence of autoregressive 
coefficients of the original process are also in ii (Q), i.e., 



< oo 



k=l 



Then, for any e > 0, there exists an integer Nq such that 



E i^^^i < I 

k=p+l 



for all p> Nq. Define N* = max{iV, iVo}- Then for all p> N*, 



E 

k=l 



— -Ok 



a, 



< e 



Therefore, 



lim > 



p^oo 



k=l 



bk,f 



bk 



It follows from the triangle inequality that 



E 

k=l 



bk,p — bk 



p 

£ E 

k=l 
P 

< 

k=l 



E 



bk,p 



Ok,p 

-I 



bk 



bk. 



+ E 



bu 



bk 



+ 



k=l 

al-l 



Ei^^i 



Therefore, as p — )• oo. 



lim y 



k=l 



bk,p - bk 



p 

< lim y 

k=l 



p k=l 



-^-bk 



+ lim — > \bk 
p-yoo a„ ^-^ 
fc=i 

By ( 14.91 ). the first limit on the right hand side is zero and by remark |4~T1 



Therefore, 



lim cj„ = 1 



lim l&fcp - = 



fc=l 



(4.9) 



□ 



Proposition 4.1. Lef 5x(A) > 0/or A G ("^i 2] '^"^ Xlfcez l-^^l < as p oo, the spectral 



density o/]E[X„|i7^_^] = X^^p converges to that o/]E[X„|ff„_i] at the origin. 
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Proof. Let {Rk,p]k&, {Rk}k& be the covariance sequences of ]E[X„|i7^_]^] , ]E[X„|i7„_i] respectively. If 
(A), S^{\) be the spectral densities of the two processes respectively, then 



2'Ki\k 



-2TTiXk 



kez 
fcez 

At the origin, i.e., at A = 0, the above spectral densities are given by 



Sj^iO) = ^Rk 

kGZ 



For some p and k, Rk,p may be obtained from ( 12.51) as 



R. 



k,p 



^[Xn,pXn~k,p\ 

P P 
E ^ bj^pXn-j ^ bipXn-k-l 
J=l 1=1 
p p— 1 p—i 

b'j,pRk + ^ ^ bj^pbj+t,piRk-t + -Rfc+i) 



Summing the above over all A; G Z gives 



) p—i p—t 

,j=i t=i j=i J k& 

,i=i / feez 



From where it follows that 



(4.10) 



(4.11) 



hm Rk^p = lim 6j- p 



p— ^-oo 



fcez \j=i / fcez 

Proceeding similarly, using the expression in (12.11 ). one obtains the following expression for ^ 



(4.12) 



Rk 



fcgZ \ j,=l / fcgZ 



(4.13) 
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Recall that by lemma |42l under the stated conditions, 

p 

lim V|6fc,p-6fc| =0 (4.14) 



k=l 

Clearly, then, 

p 

lim -h) = (4.15) 



k = l 

i.e.. 



p 



Combining (14.161 ) with (14.121) . and comparing with (14.131 ). we obtain 



\fc=l / fcGZ 

^flfc (4.17) 



This completes the proof. 



□ 



Remark 4.2. Combining the above result with the fact that Vn,p converges to Vn in quadratic mean it readily 
follows that the spectral density of{Xn,p + J^n,p} converges to the spectral density of{Xn} at the origin, i.e., 
at X = 0. 

4.2 Convergence of the Spectral Density in L2 

We now present a sufficient condition for the L2 convergence of the spectral density of the autoregressive 
approximation as p — )• 00. 

Proposition 4.2. Let 5a- (A) > Ofor AG (-^, |] and let 

|afc| < 00 (4.18) 



Then as p ^ 00, Sj^ (A) converges to S'-^(A) in L2. 

Proof. We begin by noting that when the sequence {a^} is in ii, (as stated above), both {Rk} and {6^} are 
also in £1 and therefore the conditions of Lemma |42] are satisfied. 

Refer to equation ()4.10p for an expansion of Rk^p for each k, for a given p: 

p P~l P~i 

Rk,p = ^ ^i,p-^fe + E E ^j,p^j+t,pi^k-t + -Rfe+i) 
j=l t=l j=l 

Consider the WSS process given by J2^=i bk^n-k and let {Rk,p} be its covariance sequence. Proceeding 
as in the case of ( 14.101 ). we can obtain a similar expression for R^^p as follows. 

p p— 1 p^t 

Rk,p = Yl + E E bjbj+t{Rk~t + Rk+t) (4.19) 
j=i t=i i=i 
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so that for all G Z 



p— 1 p—t 

i=i j=i 
p 

p-i p-t 

+ - bj,P^j+t,p)\ i\Rk~t\ + \Rk+t\) 

t=l j=l 



Summing over all k ^ Z 



i-Rfc,p — Rk,p\ 

fcgZ 

+2 E E - hpbj+t,p\ )^\Rk\ 

t=l 7 = 1 ^ fegz 



p p 



Yl ~ ^*.p^i.pi Y i-^fci 

p p p p 



Y E 1^*1 i^i - hp\ + E E \hp\\b^ - Kp\ ) E i^'^i 

j=l i=l j=l i=l ^ k<3L 

= ( E i^^i + E i^^.^'i I ( E - ^^^'1 1 E i^^i 



vi=l i=l 



(4.20) 



(4.21) 



As the covariance sequence has been assumed to be in £i, the sequence {bk} is also in ii (171) and by Lemma 
(14.2b . as p — )• oo the second term goes to 0. Finally, by the same lemma, 



Therefore, 



lim y~] \bi.p\ < lim \bi\ 
1=1 i=l 

lim Y] \Rk.p - Rk,p\ = 



(4.22) 



fcez 
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Now note that 



Therefore, 



Rk,p — — Vn — ^ biXn-i){XnJ^k 

j=p+l 

oo 
j=p+l 

oo oo 

= Rk— ^ bjRk-j — ^ biRk+i 
j=p+l i=p+l 



bj'E[Xn+k-j'^n] 



j=p+l 



+ ^ &i]E[X„_it'„+fc] 



i=p+l 



+ E 



biXn~i j I ^ bjXn+k- 
,i=p+l / \i=p+l 



— -Rfc — ^ bjRk-j — ^ biRk+i 

j=p+l i=p+l 
oo / oo \ 

i=p+l \i=p+l / 

oo oo 

+ y^ y^ bibi+t{Rk+t + Rk-t) 

t=l i=p+l 



\Rk,p — Rk\ 

oo oo 

< J2 N\Rk-j\+ E MRk+i\ 

j=p+l i=p+l 



Summing over all /c G Z, 



j=p+i yi=p+i 

oo oo 

+ E E Mbi+t\{\Rk+t\ + \Rk^ 

t=l i=p+l 
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\ Rk,p — Rk\ 



< 



, j=p+l / fcgZ 



+ E N EK1-.1+ E nMEi^^ 



I j=p+l / fegZ 



+ (e E (ei^''+*i+ei^'^-*i) 



(4.24) 



, j=p+l 



A;=0 



+ E i^'i Ei^^^i 



As p — )• 00, each term on the right hand side of the above inequality goes to zero, because the covariance 
sequence and the sequences and {a^} are in ii. Therefore, 



lim \Rk,p - Rk\ =0 

Combining the results of ( I4.22I ) and ( |4.25b we obtain 

lim \Rk p - Rk\=Q 

p— >-oo ^ ' 



(4.25) 



(4.26) 



Finally, 



lim 11% (A)-5^(A)|| 



p— >oo 



lim 

p— >-oo 



< lim 

p->oo 



A: 62 



dX 



< lim - Rk\ 

Vfegz / 



(4.27) 



where (14.261 ) is used for the last equality. This completes the proof. 



□ 



Remark 4.3. Combining the above result with the fact that i'n,p converges to Vn in quadratic mean it readily 
follows that the spectral density of{Xn,p + Vn,p} converges to the spectral density of {Xn} in L2. 
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5 Conclusion 



In this paper it has been shown that the spectral density of a finite autoregressive approximation of a WSS 
process converges at the origin when the spectral density is strictly positive. Thus, any unbiased spectral 
estimator derived from a finite autoregressive approximation will converge to the spectrum of the original 
process at the origin. This would enable easy approximation of the TAVC, which is important in the context of 
steady-state simulation. 

Furthermore, it has been shown that the spectral density of both the moving average and the autoregressive 
type approximations converge in L2 when the sequence of the Wold expansion coefficients is in ii. For a zero 
mean wide sense stationary processes having an infinite order moving average representation, the condition 
Ylo<k<oo \ < 00 is met when the covariance Rk tends to zero at an exponential rate as /c — )■ 00 asymptoti- 
cally; i.e., there exist constants C G M, q G (0, 1) such that \Rk\ ~ Cal'^' (I®). A more trivial example is that 
of a process which has a finite order moving average representation. In such cases, the spectral density of the 
original process can be approximated over AG [— ^, ^] from that of the finite order estimate. 
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